The Multivariate Resultant is NP-hard 
in any Characteristic 



Bruno Grenet, Pascal Koiran, and Natacha Portier* 

LIP**, Ecole Normale Superieure de Lyon, Universite de Lyon 
and Department of Computer Science, University of Toronto 
{Bruno . Grenet , Pascal . Koiran, Natacha. Portier }@ens-lyon. f r 



Rapport de Recherche RRLIP2009-34 



Abstract. The multivariate resultant is a fundamental tool of computational algebraic ge- 
ometry. It can in particular be used to decide whether a system of n homogeneous equations 
in n variables is satishable (the resultant is a polynomial in the system's coefficients which 
vanishes if and only if the system is satisfiable) . In this paper we present several NP-hardness 
results for testing whether a multivariate resultant vanishes, or equivalently for deciding 
whether a square system of homogeneous equations is satisfiable. Our main result is that 
testing the resultant for zero is NP-hard under deterministic reductions in any characteristic, 
for systems of low-degree polynomials with coefficients in the ground field (rather than in an 
extension). We also observe that in characteristic zero, this problem is in the Arthur-Merlin 
class AM if the generalized Riemann hypothesis holds true. In positive characteristic, the best 
upper bound remains PSPACE. 



1 Introduction 

Given two univariate polynomials, their Sylvester matrix is a matrix built on the coefficients of 
the polynomials which is singular iff the polynomials have a common root. The determinant of 
the Sylvester matrix is known as the resultant of the polynomials. This determinant is easy to 
compute since the size of the Sylvester matrix is the sum of the degrees of the polynomials. 
The study of the possible generalizations to multivariate systems comes within the scope 
of the theory of elimination [32,26,9,31,10]. This theory proves that the only case where a 
unique polynomial can testify to the existence of a common root to the system is the case of 
n homogeneous polynomials in n variables: the resultant of a square system of homogeneous 
polynomials fi,...,f n € K[ polynomial in the indeterminate coefficients of 

fi , . . . , f n which vanishes iff /i , . . . , /„ have a nonzero common root in the algebraic closure of 
K. The resultant of such a system is known as the multivariate resultant in the literature. This 
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captures the case of two univariate polynomials modulo their homogenization. Furthermore, in 
many cases a system of more than n homogeneous polynomials in n variables can be reduced 
to a system of n homogeneous polynomials, so that the square case is an important one. This 
result is sometimes known as Bertini's theorem (as explained toward the end of this section, 
we will use an effective version of this result in one of our NP-hardness proofs). In this paper, 
we focus on the multivariate resultant which we simply refer to as the resultant. 
The resultant has been extensively used to solve polynomial systems [25,28,5,7] and for the 
elimination of quantifiers in algebraically or real-closed fields [29,15]. More recently, the mul- 
tivariate resultant has been of interest in pure and applied domains. For instance, the problem 
of robot motion planning is closely related to the multivariate resultant [3,6], and more gener- 
ally the multivariate resultant is used in real algebraic geometry [4,20]. Finally, in the domain 
of symbolic computation progress has been made for finding explicit formulations for the 
resultant [19,8,2,7,16], see also [18]. 

Definition 1. Let K be afield and fi, . . . ,f n ben homogeneous polynomials in K[xi, . . . , x n ], 
fi(x) = ^2\ a \=d li,a% a ■ The multivariate resultant R of fi,...,f n is an irreducible polyno- 
mial in K\ji^] such that 

R{—) = 3x€KJi{x) = --- = f n (x) = 0. (1) 

The multivariate resultant is unique up to a constant factor. 

The problem we are interested in is testing the resultant for zero. This is the same as deciding 
whether a square system of homogeneous polynomials (that is n polynomials in n variables) 
has a non-trivial root. This is closely related to the decision problem problem for the exis- 
tential theory of an algebraically closed field. This problem is sometimes called the Hilbert 
Nullstellensatz problem: 

Definition 2. Let K be afield andK be an algebraic closure o/K. The Hilbert Nullstellensatz 
problem over K, HN(K), is the following: Given a system f of s polynomials in K[xo, • • • , x n ], 
does there exist a root of f in K n+1 ? 

Let us now assume that the s components of f are homogeneous polynomials. Then the ho- 
mogeneous Hilbert Nullstellensatz problem over K, H 2 N(K), is to decide whether a non trivial 
(that is, nonzero) root exists in K. 

// / is supposed to contain as many homogeneous polynomials as variables, the problem is 
called the square homogeneous Hilbert Nullstellensatz over K, H 2 N D (]K). 

In the case of the field Q, it is more natural to have coefficients in Z. We shall use the notations 
HN, H 2 N and H 2 N D for this case where the system is made of integer polynomials. In the sequel, 
for any prime number p, the finite field with p elements is denoted by F p . The notation is 
extended to characteristic zero, and Fo = Q. 

In the case of polynomials with coefficients in Z, Canny [3] gave in 1987 a PSPACE algorithm 
to compute the resultant. To the authors' knowledge, this is the best known upper bound. 
In this paper we show that testing the resultant for zero is NP-hard in any characteristic. In 
other words, H 2 N(K) is NP-hard for any field K. 

Main results and proof techniques 

In Sect. 2 we observe that for polynomials with integer coefficients, testing the resultant 
for zero is a problem in the Arthur- Merlin (AM) class. This result assumes the generalized 
Riemann hypothesis, and follows from a simple reduction to the Hilbert Nullstellensatz. For 
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this problem, membership in AM assuming GRH was established in [21]. The remainder of 
the paper is devoted to hardness results. 

In characteristic zero, it seems to be a "folklore" result that testing the resultant for zero is 
NP-hard. We give a proof of this fact in Sect. 3 since we have not been able to find one in 
the literature. In fact, we give two proofs of two results of incomparable strength. The first 
proof is based on a reduction from the Partition problem [13, problem SP12]. The second 
proof is based on a result of Plaisted [27] and shows that the problem remains NP-hard for 
systems of only two homogeneous polynomials. For the latter result to be true, we need to 
use a sparse encoding for our two polynomials (their degree can therefore be exponential in 
the input size). 

The first proof does not carry over to positive characteristic since the NP-hardness of 
Partition relies in an essential way on the fact that the data are integers (in fact, in any 
finite field the analogue problem can be solved in polynomial time by dynamic programming) . 
Plaisted's result can be adapted to positive characteristic [33,17] but this requires random- 
ization. By contrast, our ultimate goal is NP-hardness for deterministic reductions and low 
degree polynomials. We therefore need to use different techniques. Our starting point is a 
fairly standard encoding of 3 — SAT by systems of polynomial equations. Using this encoding 
we show at the beginning of Sect. 4 that deciding the existence of a nontrivial solution to a 
system of homogeneous equations is NP-hard in any characteristic. The resulting system has 
in general more equations than variables. In order to obtain a square system we explore these 
two basic strategies: 

(i) Decrease the number of equations. 

(ii) Increase the number of variables. 

In Sect. 4.1 we give a randomized NP-hardness result based on the first strategy. The idea is 
to replace the initial system by a random linear combinations of the system's equations (the 
fact this does not change the solution set is sometimes called a "Bertini's theorem"). 
In Sect. 4.2 we use the second strategy to obtain two NP-hardness results for deterministic 
reductions. The main difficulty is to make sure that the introduction of new variables does 
not create spurious solutions (we do not want to turn an unsatisfiable system into a satisfiable 
system). Our solution to this problem can be viewed as a derandomization result. Indeed, it 
can be shown that the coefficients of the monomials where the new variables occur could be 
chosen at random. It would be interesting to find out whether the proof based on the first 
strategy can also be derandomized. 

2 The resultant lies in AM 

In this section we show that testing the resultant for zero is reducible to HN(K). In the case 
K = Z, this allows us to conclude (under the Generalized Riemann Hypothesis) that our 
problem is in the polynomial hierarchy, and more precisely in the Arthur-Merlin class. In 
fact, we show that this applies more generally to the satisfiability problem for homogeneous 
systems (recall that testing the resultant for zero corresponds to the square case). 

Proposition 1. For any field K, the problem H 2 N(K) is polynomial-time many-one reducible 
to HN(K). 

Proof. Consider an instance 5 of H 2 N(K), that is s homogeneous polynomials 
fi,...,f s £ K[xi, . . . ,x n ]. The polynomials fi,...,f s can be viewed as elements of 
K[xi, . . . ,x n ,yi, ■ ■ ■ ,y n ] where yi, . . . , y n are new variables which do not appear in the fi. 
Let T be the system containing all the fi and the new (non-homogeneous) polynomial 
5Z"=i X 'V' ~ 1- This is an instance of the problem HN(K). It remains to prove that S and 
T are equivalent. 
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Given a root (01, . . . , a n , 61, ■ ■ ■ , b„) of T, the new polynomial ensures that there is at 
least one nonzero a;. So (01, ...,o„) is a non trivial root of S. Conversely, suppose that 
S has a non trivial root (ai,...,a n ), and let i be such that a, 7^ 0. Then the tuple 
(01, . . . , a„, 0, . . . , 0, a^ 1 , 0, . . . , 0) where a" 1 corresponds to the variable ?/, is a root of T. 
Thus H2N(K) is polynomial-time many-one reducible to HN(K). 

Koiran [21] proved that HN 6 AM under the Generalized Riemann Hypothesis. We denote 
here by AM the Arthur-Merlin class, defined by interactive proofs with public coins (see [1]). 
Thereby, 

Corollary 1. Under the Generalized Riemann Hypothesis, H 2 N is in the class AM. 

In positive characteristic, the best upper bound on the complexity of the Hilbert Nullstellen- 
satz known to this day remains PSPACE (in particular it is not known whether the problem 
lies in the polynomial hierarchy, even assuming some plausible number-theoretic conjecture 
such as the generalized Riemann hypothesis). 

3 The resultant is NP-hard in characteristic 

We now give our first NP-hardness result, for the satisfiability of square systems of homoge- 
neous polynomial equations. As explained in the introduction, this seems to be a "folklore" 
result. We give the (short) proof since finding an explicit statement (and proof) of this re- 
sult in the literature appears to be difficult. The second part of the theorem shows that the 
problems remains NP-hard even for systems with small integer coefficients (i.e., coefficients 
bounded by 2). This is achieved by a standard trick: we introduce new variables in order to 
"simulate" large integers coefficients. It is interesting to note, however, that a similar trick 
for reducing degrees does not seem to apply to the resultant problem (more on this after 
Theorem 2). 

Theorem 1. The problem foN of deciding whether a square system of homogeneous poly- 
nomials with coefficients in Z has a non trivial root is NP-hard. 

The problem remains NP-hard even if no polynomial has degree greater that 2 and even if the 
coefficients are bounded by 2. 

Proof. The reduction is done from Partition which is known to be NP-hard [13, problem 
SP12]: Given a finite set S and a non negative integer weight w(s) for each s £ S, the problem 
is to decide the existence of subset S' such that X^seS' w { s ) = S s ^s' w ( s )- That is, the aim 
is to cut A into two subsets of same weights. 

Given such an instance of Partition where S = {si,...,s„}, let us define a system of 
polynomials. For 1 < i < n, fi(x) = Xq — xf . And 



A tuple (ao, . . . , a n ) has to verify ai = ±ao for each i to be a solution, hence the only case to 
consider is ao = 1 and Oj = ±1 for i > 1. Then it is clear that the system has a non trivial 
solution iff S may be split into two subsets of equal weights. 

For the second part of the theorem, it remains to show that the coefficients in the system can 
be bounded by 2. As the w(si) may be big integers, they have to be replaced by variables. 
Let us write w(si) = Y^=o Wij2 J . For each Wij, a new variable Wij is introduced. For every 
i, the values of the Wij are defined by a descending recurrence: 



/o(xo, . . . ,x n 



) 



w(si)xi + w(s 2 )x 2 + 



■ + w(s n )x n . 



(2) 




(3) 
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These equalities imply that for every i,j we have Wij = Y^n=j Wij2 j ~ l xo- Then fo is replaced 
by WifiXi + \V2fiX2 + ■ ■ ■ + W n fiX n . Doing so, the number of polynomials remains the same 
as the number of variables. Hence, this algorithm build a new homogeneous system where 
the polynomials have their coefficients bounded by 2 and their degrees too. One can readily 
check that the new system has a non trivial solution iff the original one has. In particular, if 
xo is set to zero, then all other variables have to be zero too. □ 

A related result is Plaisted's [27] on the NP-hardness of deciding whether the gcd of two sparse 
univariate polynomials has degree greater than one. By homogenization of the polynomials, 
this is the same problem as in Theorem 1 for only two bivariate polynomials. Note that the 
polynomials are sparse and can be of very high degree since exponents are written in binary 
(this polynomial representation is sometimes called "supersparse" [17]). If both polynomials 
were dense, the resultant could be computed in polynomial time since it is equal to the 
determinant of their Sylvester matrix. Plaisted's theorem stated in the same language as 
Theorem 1 is the following: 

Theorem 2. Given two sparse homogeneous polynomials in Z[x,y], it is NP-hard to decide 
whether they share a common root in C 2 . 

We briefly sketch Plaisted's reduction since it will help understand the discussion at the end 
of this section. For a full proof (including a correctness proof), see [27, Theorem 5.1]. 

Proof (sketch). The idea is to turn a 3 — SAT instance into a system of two univariate poly- 
nomials which share a common root iff the 3-CNF formula is satisfiable. 
To every variable Xj is associated a prime pj, and M = YlPj i s defined where the product 
ranges over all variables that appear in the formula. A formula <j> is turned into a polynomial P$ 
according to the following rules. A non negated variable Xj is turned into Px } (x) = x M ^ v i — 1 

and a negated variable -<X k into P^x k (x) = l+x M/Pk -\ \. x (vk-^)M/vk _ Tnen a formula 0Vi/> 

is turned into P^vy. = lcm(P^, Py,). A conjunction <t> ~ Ki^i ls turned into the polynomial 

P^{x)=x m Y j P^{x)P^{1/x) (4) 

This defines the first polynomial P. The second polynomial is simply x M — 1. The proof that 
those two polynomials share a common root iff <f> is satisfiable is omitted. 
To obtain the result in the way we stated it, it is sufficient to homogenize P(x) and x M — 1 
with the second variable y. □ 

Theorems 1 and 2 seem to be incomparable. In particular, it is not clear how to derive 
Theorem 1 from Theorem 2. A natural idea would be to introduce new variables and use 
the repeated squaring trick to reduce the degrees of the polynomials occurring in Plaisted's 
result. However, as we now explain this can lead to the creation of unwanted roots at infinity 
in the resulting polynomial system. 

Assume for instance that we wish to get rid of all occurrences of x 2 in a polynomial. One can 
add a new variable X2, replace the occurrences of x by xi and add a new polynomial X2 — x . 
In order to keep the system homogeneous, the idea is to homogenize the latter polynomial: 
X0X2 — x 2 . The problem with this technique is that it adds some new roots with all variables 
but X2 set to 0, and in particular the homogenization variable xo- 

To give an explicit example of the problem mentioned above, let us consider the formula 

(XVY) A(-.X) A(-.Y). (5) 

Let us associate the prime number 2 to the variable X, and 3 to Y (M in the previous proof 
is therefore 6). By Plaisted's construction, X is turned into 

x M/2 _ 1 = x . i _ 1 

and Y into 
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x 2 — 1. Their negations -^X and ->Y are respectively turned into 1 + a; 3 and 1 + x 2 + x 4 . The 
disjunction of X and Y is turned into the 1cm of x [i — 1 and x 2 — 1, that is (x 2 — l)(x 2 + x + 1). 
Finally, we have to apply formula (4) with the latter polynomial, 1 + a; 3 and 1 + x 2 + x 4 . 
Therefore, the two polynomials of Plaisted's construction are x M — 1 = x e> — 1 and —x 3 + 
x 4 + 2x 5 + 9x 6 + 2x 7 + x s — x 9 . It can be checked that as expected, those two polynomials 
do not share any common root. 

Applying the repeated squaring trick with homogenization on this example gives the following 
system where the two first polynomials represent the original ones and the other ones are new 
ones: 

—xz + X4 + 2xb + 9a; 6 + 2x 7 + X'g — xg = 
xe, — xo = 0; X0X2 — x 2 — 0; X0X3 — X2X — , . 

X0X4 — x\ = 0; X0X5 — X4X — 0; xoXq — X2X4 — 
s X0X7 — X4X3 = 0; xox$ — X4 — 0; xgxg — x%x — 
But in that example, one can easily check that solutions with xo — exist. Namely if we set 
xg and xg to the same nonzero value and all other variables to 0, this defines a solution to 
the system. 

To the authors' knowledge, there is no solution to avoid these unwanted roots. Furthermore, 
Plaisted's result works well with fields of characteristic 0, but as it uses the fact that a sum 
of non negative terms is zero iff every term is zero, this generalizes not so well to positive 
characteristic. In particular, generalizations to positive characteristic require randomization 
(see [17] and [33]). By contrast, two of the reductions given in the next section are determin- 
istic and they yield systems with polynomials of low degree (i.e., of linear or even constant 
degree). 



4 The resultant is NP-hard in arbitrary characteristic 

In this section we give three increasingly stronger NP-hardness results for testing the resul- 
tant. As explained in the introduction, we first provide in Sect. 4.1 a NP-hardness proof for 
randomized reductions. We then give in Sect. 4.2 two NP-hardness results for deterministic 
reductions: the first one applies to systems with coefficients in an extension of the ground 
field, and the second (stronger) result to systems with coefficients in the ground field only. 
The starting point for these three NP-hardness results is the following lemma. 
Lemma 1 ([23]). Given a field K of any characteristic, it is HP -hard to decide whether a 
system of s homogeneous polynomials in K[xo, • • • , x n ] has a non trivial root. That is, H 2 N(K) 
is NP-hard. 

In [23], H 2 N(K) was proven NP-hard by reduction from Boolsys. An input of Boolsys is a 
system of boolean equations in the variables X\ , . . . , X n where each equation is of the form 
Xi — True, Xi = ->Xj , or Xi — Xj V Xk ■ The question is the existence of a valid assignment 
for the system, that is an assignment of the variables such that each equation is satisfied. 
This problem is easily shown NP-hard by reduction from 3 — SAT. We now give a proof of 
this lemma since the specific form of the systems that we construct in the reduction will be 
useful in the sequel. This proof is a slight variation on the proof from [23]. 
Proof. Let K be a field of any characteristic p, p being either zero or a prime number. At 
first, p is supposed to be different from 2. The proof has to be slightly changed in the case 
p — 2 and this case is explained at the end of the proof. 

Let B be an instance of Boolsys. Let us define a system of homogeneous polynomials from 
this instance with the property that B is satisfiable iff the polynomial system has a non trivial 
common root. The variables in the system where Xi, 1 < i < n, corresponds 

to the boolean variable Xi in Boolsys, and xo is a new variable. The system contains four 
kinds of polynomials: 
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Xq — xj, for each i > 0; 

xo ■ (xi + xo), for each equation Xi — True in Boolsys; 
xo ■ (xi + Xj), for each equation Xi — —*Xj\ 

(xi + xo) 2 — (xj + xo) ■ (xk + xo), for each equation Xi — Xj V Xk- 



Let us denote by / the polynomial system obtained from B. The first kind of polynomials 
ensures that if (oo, . . . , a n ) is a non trivial root of /, then a§ = a? = • • • = a n . Now if / has 
a non trivial root (oo, . . . ,%), then one can readily check that the assignment Xi = True 
if o» = — oo and Xi — false if a, = do satisfies B. Conversely, if there is a valid assignment 
Xi, . . . , X n for B, any (n + l)-tuple (oo, . . . , a„) where ao / and a, = — oo if Xi = True 
and o» = ao if Xi = false is a non trivial root of /. 

This proof works for any field of characteristic different from 2. The problem in characteristic 
2 is the implementation of Boolsys in terms of a system of polynomials. Indeed, for the other 
characteristics, the truth is represented by — ao and the falseness by ao- In characteristic 2, 
those values are equal. Yet, one can just change the polynomials and define in the case of 
characteristic 2 the following system: 

— xoxi — x\, for each i > 0; 

— xo(xi + xo), for each equation Xi = True in Boolsys; 

— xo(xi + Xj + xo), for each equation Xi = ->Xj; 

— + XjX k + xo ■ (xj + Xk), for each equation X, = Xj V X k . 

Now, given any nonzero value ao for xo, the truth of a variable Xi is represented by Xi = ao 
whence the falseness is represented by x% — 0. A root of the system is in particular a root of 
the polynomials defined by the first item. Therefore each Xi has to be set either to a or to 
0. The system has a non trivial root iff the instance of Boolsys is satisfiable. □ 

4.1 A randomized reduction 

We now give the first of our three NP-completeness results in positive characteristic. The 
proof also applies to characteristic zero, but in this case Theorem 1 is preferable (its proof is 
simpler and the NP-hardness result stronger since it relies on deterministic reductions). For 
more on randomized reductions, see [1]. 

Theorem 3. Let p be either zero or a prime number. The following problem is NP-hard under 
randomized reductions: 

— INPUT: a square system of homogeneous equations with coefficients in a finite extension 



- QUESTION: is the system satisfiable in the algebraic closure of¥ p ? 
In the case p — 0, the results also holds for systems with coefficients in Z. 

Proof. Lemma 1 shows that it is NP-hard to decide whether a non square polynomial system / 
with coefficients in F p has a non trivial root. From /, a square system g is built in randomized 
polynomial time. 

Let us denote by fj, 1 < j < s, the components of /. They are homogeneous polynomials in 
¥ p [xo, ■ ■ ■ , x n \. The components of g are defined by 



for < i < n. In the sequel, we explain how to choose the ctij for / and g to be equivalent. 
For any choice of the ctij , a root of / is a root of g. Thus it is sufficient to show how to choose 
the ctij so that g has no non trivial root if the same is true for /. 



of¥ p . 



S 




(7) 
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The property the otij have to verify is expressed by a first-order formula: 

*(a) = Vxo (^/\ fj (x) = j V (y E a ^ fi (*) • °) • ( 8 ) 

The formula $ belongs to the language of the first-order theory of the algebraically closed 
field F p . This theory eliminates quantifiers and <P(a) is therefore equivalent to a quantifier- free 
formula of the form 



= V l/\Pki(a) = OA/\Q km (a)^o\ , 

k \ I m / 



(9) 



where PuuQkm G F p [a]. As a special case of [11, Theorem 2], one can bound the number of 
polynomials in <P as well as their degrees by 2 p ( n ' log ( s+n " where P is a polynomial independent 
from $ . 

The proof of Theorem 6 (in Appendix) shows that the set A of tuples satisfying $ is Zariski- 

s(n+l) 

dense in F p . Since A is dense, and A is also defined by ty, one of the clauses of must 
define a Zariski dense subset of F p . This clause is of the form /\ Q m (a) =^ 0. 
To satisfy <£, it is sufficient for the ouj to avoid the roots of a polynomial Q = Y[ m Qm- As 
mentioned before, it is known that 'P contains at most 2 p ( n ' log ( s+n ^ polynomials of degree 
at most 2 p ("' lo s( s +™». Thus, Q is a polynomial of degree at most 2 2P ^' 1 ° s< - s+n » . Consider 
now a finite extension K of F p with at least 2 2+2P ' n ' log ( s+n ^ elements (that is, of polynomial 
degree). If we choose the ctij uniformly at random in K, then with probability at least 3/4 
they are not a root of Q (by the Schwartz-Zippel Lemma). Thus with the same probability, 
they satisfy <P. Note that K can be built in polynomial-time with Shoup's algorithm [30] when 
p is prime (for p — 0, we take of course K = Q). 

To sum up, we build from / a square system g defined by random linear combinations of the 
components of /. If / has a non trivial root, then it is a root of g too. Conversely, if / has no 
non trivial root, then with probability at least 3/4 it is also the case that g has no nontrivial 
root. □ 

In characteristic zero the bounds in the above proof can be sharpened: instead of appealing 
to the general-purpose quantifier elimination result of [11] we can use a result of [24]. Indeed, 
it follows from section 4.1 of [24] that there exists a polynomial F of degree at most 3 n+1 
such that F(a) ^ implies that g has no non trivial root as soon as it is true for /. This 
polynomial plays the same role as Q in the previous proof but the bound on its degree is 
sharper. 



4.2 Deterministic Reductions 

We now improve the NP-hardness result of Sect. 4.1: we show that the same problem is NP- 
hard for deterministic reductions. This result is not only stronger, but also the proof is more 
elementary (there is no appeal to effective quantifier elimination). 

Theorem 4. Letp be either zero or a prime number. The following problem is HP -hard under 
deterministic reductions: 

- INPUT: a square system of homogeneous equations with coefficients in a finite extension 
o/F p . 

- QUESTION: is the system satisfiable in the algebraic closure of¥ p ? 
In the case p = 0, the results also holds for systems with coefficients in Z. 
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Proof. The proof of Lemma 1 gives a method to implement an instance of Boolsys with a 
system / of s homogeneous polynomials in n + 1 variables with coefficients in ¥ p . It remains 
to explain how to construct a square system g that has a non trivial root iff / does. Let us 



denote by /i , . . . , f a the components of /, with for each i — 1 , . . . , n, fi = x 



lip + 2 



and fi = xqX{ — x% if p = 2. A new system g of s polynomials in s variables is built. The s 



variables and , 

system g is the following: 



, Us-n-i, that is (s — n — 1 ) new variables are added. The 



/A (2) 



g(x,y) 



fn+l(x) 
fn+2(x) -yl 



fn+i{x) 



2 



+A2/I 



fs-i(x) -y 2 s 

\fs(x) 



2 +Ay. 



s — n—1 



(10) 



/ 



The parameter A is to be defined later. Clearly, if / has a non trivial root a, then (a, 0) is a 
non trivial root of g. Let us now prove that the converse also holds true for some A: if g has 
a non trivial root, then so does /. Note that a suitable A has to be found in polynomial time. 
Let (ao, • • • , a n , &i, . . . , b s - n -i) be any non trivial root of g. Since a must be a common root 



of /i, . . . , /„, we have a 



a„ if p / 2, and a,i £ {0, ao} for every i if p — 2. Now, either 



ao — and fi(a) = for every i, or ao can be supposed to equal 1. Therefore, if p ^ 2 either 
a = or a; = ±1 for every i, and if p = 2 either o = or a ; £ {0, 1} for every i. Let us define 
£j = /»+<(«) £ F p . As (a, 6) is a root of <j, the &f satisfy the linear system 



£2 - 

£s — n — 1 ^s- 



Fi 



AFi 
AF 2 



n-2 + \Y S - 
n-1 



o, 
o, 



-1 = 0, 
= 0. 



(11) 



This system can be homogenized by replacing each by e,Yo where lo is a fresh variable. 
This gives a square homogeneous linear system. The determinant of the matrix of this system 
is equal to (-l)*-™- 1 (ei + e 2 A + ■ ■ ■ + e^A 3 "™- 1 ). 

Let us consider this determinant as a polynomial in A. This polynomial vanishes identically 
iff all the e, are zero. In that case, the only solutions satisfy Yj = for i > 0, that is (a, 0) is 
a root of g and therefore a is a root of /. If some are nonzero, this is a nonzero polynomial 
of degree (s — n — 1). If A can be chosen such that it is not a root of this polynomial (for 
any possible nonzero value of e), then the only solution to the linear system is the trivial one. 
This means that the only non trivial root of g is (a, 0) where a is a root of /. 
If the polynomials have coefficients in Z, A = 3 (or any other integer A > 2) satisfies the 
condition. Indeed, one can check that ei = f n +i(a) G {—4,0,2,4} when ao = 1. The deter- 
minant is zero iff e[ + e' 2 X H h e' a _„A s_ ° _1 = where e- = e;/2 G {-2, 0, 1, 2}. For each 

i, let e+ = max{ei,0} and = max{— 4,0}. Then t\ = ef — e~, and < e\ ^1 < 2. Now 
the determinant is zero iff X^i e ^3 l = Si e r3'- By the unicity of base-3 representation, this 
means that for all i, ef = er, and so e\ = 0. 
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For a field of positive characteristic, this argument cannot be applied. The idea is to find a 
A that is not a root of any polynomial of degree (s — n — 1). Nothing else can be supposed 
on the polynomial because if F p = F3 for example, any polynomial of F P [A] can appear. This 
also shows that A cannot be found in the ground field. Suppose an extension of degree (s — n) 
is given as ¥ P [X]/(P) where P is an irreducible degree- (s — n) polynomial with coefficients in 
F p . Then a root of P in ¥ P [X]/(P) cannot be a root of a degree-(s — n — 1) polynomial with 
coefficients in F p . Thus, if one can find such a P, taking for A the indeterminate X is sufficient. 
For any fixed characteristic p, Shoup gives a deterministic polynomial-time algorithm [30] that 
given an integer N outputs a degree- N irreducible polynomial P in F P [X]. Thus, the system g 
is now a square system of polynomials in (¥ P [X]/(P)) [x, y] and this system has a non trivial 
root iff / has a non trivial root. And Shoup's algorithm allows us to build g in polynomial 
time from /. 

For any field F p , it has been shown that from an instance B of Boolsys a square system g 
of polynomials with coefficients in an extension of F p (in Z for integer polynomials) can be 
built in deterministic polynomial time such that g has a non trivial root iff B is satisfiable. 
This shows that the problem is NP-hard. □ 

The previous result is somewhat unsatisfactory as it requires, in the case of positive character- 
istic, to work with coefficients in an extension field rather than in the ground field. A way to 
get rid of this limitation is now shown. Yet, a property of the previous result is lost. Instead of 
having constant-degree (even degree-2) polynomials, our next result uses linear-degree poly- 
nomials. It is not clear whether the same result can be obtained for degree-2 polynomials (for 
instance, as explained at the end of Sect. 3 reducing the degree by introducing new variables 
can create unwanted solutions at infinity). 

The basic idea behind Theorem 5 is quite simple (we put the irreducible polynomial used to 
build the extension field into the system), but some care is required in order to obtain an 
equivalent homogeneous system. 

Recall from the introduction that H2N D (F P ) is the following problem: 

- INPUT: a square system of homogeneous equations with coefficients in F p . 

- QUESTION: is the system satisfiable in the algebraic closure of F p ? 

Theorem 5. For any prime p, H 2 N D (F P ) is NP-hard under deterministic reductions. 

Proof. The idea for this result is to turn coefficient A in the previous proof into a variable and 
to add the polynomial P as a component of the system. Of course, considering A as a variable 
implies that the polynomials are not homogeneous anymore. Thus, it remains to explain how 
to keep the system homogeneous. 

First, the polynomial P needs to be homogenized. This is done through the variable £0 in 
the canonical way. As P(\) is irreducible, it is in particular not divisible by A. Hence, the 
homogenized polynomial P(A, xq) contains a monomial a\ d and another one /3xq where d is 
the degree of P. Hence xo is zero iff A is. 

The other polynomials have the form f n +i(x) — yf-i + Xyf. It is impossible to homogenize 
those polynomials by multiplying f n +i and y1^\ by xo (or any other variable) because then 
the variable never appears alone in a monomial, and a s-tuple with all variables set to 
but yi-i would be a non trivial solution. Moreover, in the previous proof, the fact that the 
j/i all appear with degree 2 is used to consider the system as a linear system in the yl . Thus 
replacing the monomial Aj/f by \yi does not work either. Instead, we construct the slightly 
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more complicated homogeneous system: 

(h{x) 

fn{S) 



9h(x, y, A) = 



l fn+i(x) 



7n +2 (x) - y r n +Ayr 



X fs- 
/.(«) 



(2) 



-Vs-n-1 



(12) 



/ 



Contrary to the previous proof, the yi do not appear all at the same power. Yet, all the 
occurrences of each yi have the same degree, and we shall prove that this is sufficient. 
Let us prove that if / does not have any non trivial root, then neither does gh- Some of the 
observations made for g in the previous proof remain valid. Hence, it is sufficient to prove 
that a non trivial (s + l)-tuple (a, b, €) cannot be solution of gn whenever ao = 1, b =/= and 
Oq = • • • = a„ if p ^ 2 or at € {0, do} if p = 2. By a previous remark on the polynomial P, I 
can also be supposed to be nonzero. 
So, similarly as in the previous proof, let us define e 
gu, the variable j/» only appears at the power (s — n 
and i, the tuple (a,b,£) is a root of gn iff the b°~ 



C-l 
62 



- Y-l 
_ Y 

^s — n- 



+ 
+ 



- ag _n_l / n +i(a) € F p . In the system 
i + 1). Therefore, given a value of a 
1+1 satisfy the linear system 

lYx = 
IY 2 = 



(13) 



-2 + £Y S - 
-i 



This is the same system as in the previous proof. Now if (1, 1) is supposed to be a root of 
P, as P is an irreducible polynomial of degree (s — n), £ cannot be a root of a univariate 
polynomial of degree less than (s — n) with coefficient in F p . But the determinant of the linear 
system is such a polynomial, and thus cannot be zero. This determinant is then iff all the 
e» = 0. The same arguments as in the previous proof can be used to conclude that (a,b,£) 
can be a root of g h iff a is a root of /. 

Thus, from an instance B of Boolsys, a square homogeneous system g h of polynomials with 
coefficients in the ground field F p is built in deterministic polynomial time. This system has 
a non trivial root iff B is satisfiable. The result is proved. □ 



5 Final remarks 

In characteristic zero, the upper and lower bounds on H2N D are in a sense close to each other. 
Indeed, NP = TiP C AM C II2P, that is, AM lies between the first and the second level of 
the polynomial hierarchy. Furthermore, "under plausible complexity conjectures, AM = NP" 
[1, pl57]. Improving the NP lower bound may be challenging as the proof of Proposition 1 
shows that this would imply the same lower bound for Hubert's Nullstellensatz. 
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In positive characteristic, the situation is quite different. Indeed, the best known upper bound 
for Hubert's Nullstellensatz as well as for the resultant is PSPACE. As in characteristic zero, 
the known upper and lower bounds are therefore the same for both problems. But as the 
gap between the NP lower bound and the PSPACE upper bound is rather big, these problems 
might be of widely different complexity (more precisely, testing the resultant for zero could 
in principle be much easier than deciding whether a general polynomial system is satisfiable) . 
Canny's algorithm for computing the resultant [3] involves the computation of the determi- 
nants of exponential-size matrices, known as Macaulay matrices, in polynomial space. Those 
matrices admit a succinct representation (in the sense of [12]). One can prove that computing 
the determinant of a general succinctly represented matrix is FPSPACE-complete (and even 
testing for zero is PSPACE-complete) [14]. It follows that the FPSPACE upper bound could 
be improved only by exploiting the specific structure of the Macaulay matrices in an essential 
way, or by finding an altogether different (non Macaulay-based) approach to this problem. 
As pointed out in Sect. 2, in characteristic zero a different approach is indeed possible for 
testing whether the resultant vanishes (rather than for computing it). This problem is wide 
open in positive characteristic. 

Finally, an interesting open question is whether the randomized reduction of Theorem 3 can 
be derandomized. 

Acknowledgments. We thank Bernard Mourrain and Maurice Rojas for sharing their 
insights on the complexity of the resultant in characteristic 0. 
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A Appendix 

The following result is used in the proof of Theorem 3. 

Theorem 6. Let K be an algebraically closed field and V an algebraic variety o/K n+1 defined 
by a set of homogeneous degree-d polynomials f\, . . . , f s £ K[xo, . . . ,x n ]. This variety can be 
defined by (n + 1) homogeneous degree-d polynomials g\, ...,g n +i 6 K[xo, ■ ■ ■ ,x n ]. Moreover, 
suitable gi 's can be obtained by taking generic linear combinations of the fi. That is, we can 
take g t — X)J=i ctijfj where (Qij) is a matrix of elements ofK, and the set of suitable matrices 
is Zariski-dense in K s ^ n+1 - ) . 
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Proof. A similar result is established in [22, Proposition 1] for arbitrary (possibly non- 
homogeneous) polynomials: Any algebraic variety of K n can be defined by taking n + 1 
generic linear combinations of the original equations. In Theorem 6 the polynomials are as- 
sumed to be of same degree to ensure that the linear combinations gi are homogeneous. If 
the fi are not of the same degree, the system can be transformed into an equivalent system 
where the degree of all polynomials is equal to the least common multiple of the degrees of 
the polynomials in the original system (in the application to Theorem 3, this transformation 
is not necessary since the input system is made of polynomials of equal degree). 
Let V a be the variety defined by the gi. Clearly, V C V a for any matrix a. We need to show 
that there is a Zariski-dense set of matrices a such that V a C V . 

Consider a point (ao, . . . , a n ) G K n+1 . If ao = 0, then one can define new polynomials fj and 
g~i by setting fj(xi, . . . ,x n ) = /j(0, xi,...,x n ) and §i(xi, . . . , x„) = gi(0, xi, . . . t x n ). The 
new polynomials satisfy the same linear relations, namely, we have g~i = a ijfj- 
By [22, Proposition 1], there is a Zariski-dense set of matrices a such that for any tuple 
(oi, . . . , a n ), if the §i vanish on (oi, . . . , a n ) the same is true of the fj. In this case the fj 
vanish on (0, oi, . . . , a n ), and therefore (ao, . . . , a n ) G V. 

It remains to examine the case ao 7^ 0. In this case, for any tuple (ao, 01, . . . , a„) £ V a , we have 
(1, ai/ao, . . . , a„/ao) G V a since the polynomials are homogeneous. The same argument as 
for the case ao = shows that we will have (1, ai/ao, . . . , a n /ao) G V for a in a Zariski-dense 
set (we now apply [22, Proposition 1] to the polynomials fj(xi, . . . ,x„) = fj (1, xi, . . . , x n ) 
and §i(xi, . . . , x n ) = <?i(l, xi, . . . , x n )). By appealing again to homogeneity, we can conclude 
that (ao, . . . , a„) G V. 

From the above analysis it follows that any matrix a belonging to the intersection of two 
Zariski-dense sets (corresponding to the two cases ao = and ao =fc 0) is suitable. This 
concludes the proof since a finite intersection of Zariski-dense sets is Zariski-dense. □ 



